Implicit Wiener Series Part II: Regularised Estimation

Classical Volterra and Wiener theory of nonlinear systems does not address the problem of noisy measurements in system identification. This issue is treated in the present part of the report. We first show how to incorporate the implicit estimation technique for Volterra and Wiener series described in Part I into the framework of regularised estimation without giving up the orthogonality properties of the Wiener operators. We then proceed to a more general treatment of polynomial estimators (Volterra and Wiener models are two special cases) in the context of Gaussian processes. The implicit estimation technique from Part I can be interpreted as Gaussian process regression using a polynomial covariance function. Polynomial covariance functions, however, have some unfavorable properties which make them inferior to other, more localised covariance functions in terms of generalisation error. We propose to remedy this problem by approximating a covariance function with more favorable properties at a finite set of input points. Our experiments show that this additional degree of freedom can lead to improved performance in polynomial regression.